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Abstract This paper presents a systems modelling approach to evaluating the 
success of an agroforestry extension program in Leyte, the Philippines. During the 
program, variables which are intrinsic to farmers’ socio-economic and farming 
systems were found to have influenced the uptake and acceptance of extension 
advice. Evaluation of the program therefore depended on identifying the variables 
and their interdependencies and assessing their relative influence on program out¬ 
puts. For this purpose, a systems approach which encourages breaking systems into 
component variables, but also acknowledges the context of problems, assisted 
construction of models. Using both empirical data collected during program 
activities and input from stakeholders, Bayesian Belief Network software was used 
to predict critical success factors for four aspects of the overall extension system, 
namely recruitment, use of written extension materials, farmers’ self-efficacy and 
retention of participating farmers throughout the program. A key predicted con¬ 
straint to program recruitment is farmers’ perception of harvest security and while 
this variable can be partly addressed through dissemination of information on 
harvesting legislation, title security cannot. Differing levels of farmers’ education 
result in differences in predicted reading ability, comprehension of extension lit¬ 
erature and possible misconstrual of information. The variable most critical to the 
development of farmers’ self-efficacy is extended on-farm technical assistance and 
support. 
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Introduction 

This paper describes the use of systems modelling to evaluate an agroforestry 
extension program which was undertaken over 3 years as one of the activities of the 
Australian Centre for International Agricultural Research (ACIAR) project ASEM/ 
2003/052, Improving Financial Returns to Smallholder Tree Farmers in the 
Philippines. Systems modelling was chosen because it is well suited to the 
evaluation of agroforestry extension, whereas statistical analysis is best suited to 
experimental research, which is difficult to evaluate if interventions involve human 
interaction (Rossi et al. 2004). Economic approaches may require long-term data 
collection and the usefulness of structured surveys suffers because the context of 
responses is often ignored (Pretty et al. 1996). 

An evaluation process which includes the viewpoints of stakeholders may also be 
more appropriate than approaches in which assessments are conducted by external 
experts using predetermined indicators of success (Cramb and Purcell 2001; Owen 
2006). Although the evaluation of development assistance in the past has often 
neglected the complex processes behind assistance uptake, it is no longer acceptable 
to disregard them (Henderson and Burn 2004; Henriksen and Barlebo 2007). Hence, 
systems modelling, which includes breaking problems down into component 
variables, considering interactions between them, recognising the dangers of narrow 
model boundaries and the importance of qualitative as well as quantitative data 
(Sterman 2002), matched the evaluation needs of this program. In particular, 
systems modelling permitted inclusion of the subjective human-behavioural 
variables which had been observed throughout the program to affect outputs. 

Systems modelling also permits prediction of outputs for alternative input scenarios 
and for extension programs, this is useful to identify critical success factors or 
impediments. Identification of variables which are within the capacity of program 
managers to control, allows input-dependent prediction of upper and lower limits of 
program success. If qualitative variables are included and stakeholders are involved in 
assessing their influence, models may be populated with data which reflect stakeholders’ 
‘real life’ assessment of situations. This increases the validity and reliability of models 
when they are used to predict the success of extension programs in similar situations. 

The systems modelling presented in this paper used data from an extension program 
on Leyte Island, the Philippines. The program had been conducted as case studies in 
four municipalities in which farmers were assisted to grow seedlings in home nurseries 
and establish plantations of Swietenia macrophylla (mahogany). Evidence collected 
throughout the program and from a final survey indicated that the effectiveness of 
extension activities was influenced by social, cultural and biophysical issues. 
Accordingly, a systems approach was used to provide answers to three research 
questions: 

1. What extension activities and information are appropriate to recruit smallholder 
farmers in Leyte and to develop their self-efficacy so that they are able to 
continue the establishment of trees without further extension assistance? 

2. Will an extension program which offers technical advice and assistance be 
effective in improving the uptake of timber tree establishment? 
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3. What are the constraints, opportunities and resource requirements involved in 
scaling up an extension program as described in research question 2, from the 
local to a wider level? 

The first focus of this paper is the application of systems modelling to four 
aspects of the overall extension system: contacting and recruiting farmers, the 
effectiveness of written extension materials, development of farmers’ self-efficacy 
and retention of participating farmers over the course of an extension program. The 
second focus is an evaluation of the methods adopted for the elicitation of 
information from stakeholders. A final focus is the usefulness of the models to 
supply answers to the three research questions. 


Research Method 

The extension program is reported in Baynes et al. (2008) and a precis is presented 
below. Between 2005 and early 2008, assistance was offered to farmers in Leyte to 
grow seedlings in home nurseries and establish trees. The program consisted of 
recruitment, initial training and extension assistance to farmers in the municipalities 
of Libagon, Dulag, Leyte Leyte and Bato, the Philippines). Recruitment was made 
through local government unit (LGU) officials after permission had been granted by 
the municipal mayor. The program was designed to provide assistance in two 
formats. For volunteer-farmers in the municipalities of Dulag and Libagon, 
assistance was provided via a field tour which included an overview of small-scale 
forestry. The tour was followed by extended assistance to grow seedlings and 
establish trees, mainly mahogany. For farmers in the municipalities of Leyte Leyte 
and Bato, extension assistance was restricted to the field tour, collecting seed and 
setting up a home nursery. Compared to farmers in Bato and Leyte Leyte, farmers in 
Libagon and Dulag therefore received three extra site visits, where in addition to 
assistance to set up a home nursery, they were advised how to prepare sites, plant 
and maintain trees. The rationale underpinning the delivery of two assistance 
regimes was that if farmers are capable of establishing trees once they are initially 
competent in growing seedlings, then a hypothetically scaled-up extension program 
would be more cost-effective if extended assistance is not necessary. 

Although extension activities were well received, overall recruitment was low, many 
farmers did not use the written extension materials and the number of farmers who 
initially participated, declined throughout the program. Farmers initially displayed little 
knowledge of nursery and tree-growing. Unexpected problems (e.g. persistent rain and 
predation of seedlings by chickens and rats) caused severe loss of farmers’ confidence 
where extended assistance was not provided. These problems prompted extension staff 
to identify and document variables which had influenced the success of the program. 

Identification of System Variables 

To identify variables which had affected program outputs, data were collected as 
descriptive statistics (e.g. farmer’s attendance at extension activities), trip reports, 
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Table 1 System variables which were or were not addressed in 

the extension program 

Main extension 
program activities 

Variables which were addressed 

Variables which were not addressed 

Recruitment 

Recruitment approach through LGU 
officials or advertisements 

Farmers’ understanding of harvest 
legislation 

Farmers’ perceptions of certainty of 
harvest 

Land use 

Land availability 

Security of title 

Political favouritism 

Mayoral support 

LGU employee support 

Use of written 

Language in which materials are printed 

Farmers’ reading ability 

extension materials 

Text length 

Type of material (poster or pamphlet) 
Style of graphics 

Farmers’ comprehension of written 
materials 

Farmers’ construal of the meaning 
of written materials 

Development of self- 

Competence in performing skills 

General education 

efficacy 

Understanding of underpinning theory 

Opportunity to ask questions and clarify 
issues 

Extended support to reinforce technical 
training 

Social confidence 

Attrition of 

Initial offer of assistance 

Initial self-efficacy a 

participating farmers 

Extended support to reinforce technical 
training 

Initial feasibility 

Unexpected problems 

Destruction of sites through fire, 
flood or grazing 


Self-efficacy is defined in Appendix A 


visual observations, translated conversations with farmers, and opinions supplied by 
ACIAR extension staff. For each of the main extension activities, the variables were 
classified as either those which were addressed through program activities or those 
which were not (Table 1). It became apparent that even though the variables were 
imprecise and their levels often best expressed as subjective probability distribu¬ 
tions, they could be arranged as a causal network in which parent variables 
influenced subordinate child variables through to program outputs. This prompted 
the arrangement of the variables as a causal network and construction of preliminary 
models. 

Construction of System Models 

Choice of a systems modelling approach was complicated by the inclusion of both 
quantitative and qualitative variables, and the need to predict program outputs for 
varying levels of inputs. A clear visual display was also required to facilitate easy 
comprehension of models by stakeholders who were to be asked to validate model 
structure. Using these criteria, spreadsheets, mathematical models, multi-criteria 
analysis and decision trees were not considered useful because they are difficult to 
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explain to stakeholders (Cain 2001). In addition, causal loop diagrams do not 
represent the logical flow of extension activities unless the program is conducted 
over an extended timeframe. Neither is the concept of ‘stock flow’ diagrams 
(accumulation and changes to quantities in a system) easily aligned to modelling 
imprecise qualitative influences. However, Bayesian Belief Network (BBN) 
graphical models are suitable for probabilistic modelling through their ability to 
model cause and effect within systems. 

BBNs consist of nodes (boxes) which represent system variables (each node 
having two or more classes), links which represent causal relationships and 
probabilities which quantify the chance that a node will be in a particular state given 
that its input (or parent) nodes are in particular classes. The graphical component of 
a BBN is called an influence diagram and consists of nodes and links. The 
mathematical component of a BBN uses Bayes’ formula to calculate the probability 
of the occurrence of an event (a conclusion) conditional on the occurrence of other 
events, (a premise). Variables which can be identified as causally influencing the 
outcome of other variables are linked into a network and variables further along 
the chain are modified by the influence of variables higher up. BBNs therefore show 
the logical consequences of linking the user’s understanding of parts of a system 
into an integrated whole (Jensen 2001; Cain et al. 2003). They are becoming an 
increasingly popular modelling tool in environmental management because they 
allow for the integration of biophysical, economic and social variables (McCann 
et al. 2006, Uusitalo 2007). 

In Netica™ BBN modelling software, links encode the conditional dependencies 
between variables in probability tables. Probabilities inserted into the tables may be 
derived from empirical evidence or a personal belief and must be based on 
defensible evidence and reasoning (O’Hagen 2003). Alternatively, if the evidence is 
taken from a population which is being modelled, then the frequency distribution 
implicit in that data may be used as an approximation of the desired probabilities 
(Norsys 2007). The software also provides a sensitivity analysis capability which 
calculates entropy reduction to identify those parts of a model which most affect 
output variables. 

For model construction, data were sourced from descriptive statistics which had 
been collected throughout the program and qualitative data derived from interviews, 
visual observations by ACIAR staff and feedback from local government unit 
(LGU) officials. Overall, 52 farmers initially expressed interest in the program, with 
sub-sets of farmers involved in various aspects. This provided estimates of some 
variables which were likely to be more reliable than farmers’ and LGU officials’ 
subjective opinions. However, other variables were qualitative, (e.g. farmers’ 
reading ability) and were not easily estimated. Where necessary, this prompted 
sourcing of data from stakeholders in order to populate models. 

Preliminary models of the four main extension activities were constructed 
following recommendations for model construction that models should be made as 
parsimonious as possible—capturing the main factors that link a program to its 
outcomes—without being so complex that the sensitivity of the output variable to 
input variables is swamped by intermediate variables and validation becomes 
difficult (Cain 2001; Donaldson and Gooler 2003; Marcot et al. 2006). It was not 
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possible to involve stakeholders directly in the identification of variables and the 
construction of preliminary models. Therefore, after initial development, the 
preliminary models were initially assessed by an expert group of three Filipino 
researchers, ready for formal validation by stakeholders at a workshop in Leyte. 
Definitions of key variables are described in Appendix A. 

Validating Model Structure and Populating the Models at a Workshop 

A workshop was held in Leyte in July 2008 to validate and populate the models. 
Fourteen stakeholders participated in the workshop, comprising seven extension 
program participants who actively farmed their land (farmers) and seven partic¬ 
ipants who employed farm labour rather than performing the work themselves (non¬ 
farmers). All of the non-farmer group had outside employment or held positions in 
local government. 

Validation of the preliminary models was premised on the definition proposed by 
Cain et al. (2003) that the validity of data elicited from stakeholders is contingent on 
inducing them to offer their own perspectives and participate in discussion. Because 
the results of the workshop could be compromised if shyness or reading and writing 
limitations inhibited participants from responding fully to questions, it was decided 
to employ a one-to-one ratio of assistants to workshop participants. The task of the 
assistants was to translate if necessary, recapitulate information or instructions, 
elicit comments and to record a verbatim reply to each question. Fourteen staff were 
available to assist with the workshop and the number of participants was also 
consequently set at 14. 

The workshop had two objectives, to validate the structure of the preliminary 
models and, where necessary, to populate the models. Using a simple BBN model, a 
Cebuano-speaking facilitator first demonstrated the calculation of conditional 
probability. Each of the four models was shown to the participants and the meanings 
and definitions of the variables were explicitly described. Participants were then 
asked to comment whether they thought that the models and their constituent 
variables were realistic, and an accurate representation of the influences which 
affect farmers’ motivation and engagement in small-scale forestry. They were also 
asked whether variables should be added, deleted or re-ordered in the networks. 
Finally, they were asked to assess the probability of the incidence of each variable. 

Participants were asked to indicate on a Likert scale their opinion of the 
likelihood of each variable taking a particular state and to make a written comment. 
To ensure that responses were only used when participants had understood the 
question, whenever participants did not record a comment which matched their 
response on the Likert scale, their response was treated as invalid. Valid Likert scale 
responses were averaged and the resulting mean probabilities entered into BBN 
probability tables. If two or more parent variables influenced a child variable, 
participants were asked to weight the influence of the variables in elicited 
probability tables using the procedures described by Cain (2001). 

It was possible that socio-economic differences between the farmer and 
non-farmer groups may have caused them to respond to questions differently. 
Therefore, because the sample size was small, for four variables, a non-parametric 
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Mann-Whitney rank-sum test was performed for data supplied by both groups to 
test whether the farmer and non-farmer group had responded differently. The 
variables were: 

1. On non-intensively managed farms, the probability of farmers having land on 
which to plant trees 

2. The likelihood of political favouritism affecting recruitment in municipalities 

3. Farmers’ ability to read paragraphs, written in their local dialect 

4. The probability of farmers having insecure title to their land. 

The four variables were chosen because they are examples of variables which 
were wholly or partly beyond the capacity of the extension program to control and 
because they are potentially critical impediments to the success of an expanded 
program. 


Workshop Outcomes 

Workshop participants validated the structure of three models, i.e. models of the 
recruitment of farmers to an agroforestry extension program, the effectiveness of 
extension materials and the development of self-efficacy. They also supplied data 
with which to populate the models. For a fourth model which describes the attrition 
of participating farmers throughout a program, participants validated the structure of 
the model but empirical data which had been collected during the extension 
program were used to populate probability tables. 

For most questions, the assistants were able to record a comment which indicated 
that participants understood the nature of questions. Overall, only 6.5% of responses 
were rejected because participants did not supply a valid comment. 

For the four variables to which the Mann-Whitney rank-sum test was performed 
for data supplied by the farmer and non-farmer groups, a test of probability 
estimates assigned by both groups showed that for each variable, no significant 
difference was found between the responses obtained from each group (P > 0.05, 
two-tailed test). Data from all 14 participants were therefore processed as one 
dataset. 

Result of Workshop Participants’ Input for Model 1: ‘Recruitment of Farmers 
to an Agroforestry Extension Program’ 

In preliminary questions, participants provided a demographic snapshot of a 
‘typical’ farmer in Leyte. The average age of farmers in Leyte was estimated as 
being over 50, with 35% of them having only elementary education or not having 
attended school. Approximately 30% of farmers were categorised as being poor 
tenant farmers with a consequent low level of involvement in community affairs. 

Participants suggested that the recruitment of farmers to an extension program 
should be treated first as the level of contact achieved with farmers at the municipal 
level (Model la, Fig. 1) and second as the recruitment of farmers at a personal level 
(Model lb, Fig. 2). 
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Fig. 1 BBN model of the level of initial contact with farmers achieved through LGU staff or radio and 
painted signs 



Fig. 2 BBN model of influences which affect the motivation of farmers to be recruited to an agroforestry 
extension program 


Participants accepted that contact with farmers at the municipal level is affected 
by land-use, mayoral support, employee support and political divisions (Fig. 1). 
Once farmers have been contacted, however, they considered that recruitment is 
contingent on land availability and farmers’ perceptions of harvest certainty, as 
described in Fig. 2. Harvest certainty was seen as being influenced by farmers’ 
security of title to their land and their understanding of harvest legislation. 

In Fig. 1, the predicted level of support from municipal mayors is strongly 
influenced by the intensity of land use in the municipality. If typical farms are not 
intensively cultivated, strong mayoral support is highly probable (i.e. 98%), whereas 
if land in the municipality is typically intensively harvested, agroforestry or 
protective tree planting is not a political priority and the probability of strong 
mayoral support is predicted to be only 38%. This finding mirrors the results of the 
extension program and indicates that agroforestry programs are much more likely to 
be supported by elected officials and LGU staff in situations where land is not seen 
as being profitably used for annual crops. 
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For land which is not intensively farmed and where no political tensions exist in 
the community, the predicted probability of achieving a high level of contact with 
farmers is 96%, compared to 61% where divisions exist. Using empirical data 
collected during the program, the model was constructed so that the probability of 
contact with farmers is ‘very low’ if only radio announcements or painted signs are 
used to contact farmers. Despite detailed planning and execution, recruitment 
through these media in the municipalities of Libagon and Dulag (signs) and Baybay 
(radio) had been an almost complete failure. Participants commented that although 
political favouritism is a normal factor of rural Filipino life they envisaged that the 
culturally appropriate way to gain access to communities is via elected politicians. 

For farms which are not intensively managed, participants rated the probability of 
land being available to grow small numbers of trees 1 as being 56% compared to 
38% for land which is intensively managed. On intensively managed land therefore, 
participants still saw considerable scope for small-scale tree planting. Several 
participants commented that farmers would always have a preference for growing 
annual crops, but that the security of these crops is difficult on plots of land distant 
from the family home. 

The probability of farmers having a high understanding of harvest legislation 
(25.6%) had been derived from interviews with 39 farmers during the extension 
program. In addition, at the workshop participants considered that the probability of 
farmers having security of title was only 56%, this issue being a serious impediment 
to the development of farms in the Philippines. Using these data, only 11% of 
farmers are predicted to be certain of harvesting their trees (‘harvest certainty’, 
Fig. 2). Alternatively, if an extension program provides information sessions so that 
all farmers have a high understanding of relevant legislation, the predicted 
probability of harvest certainty rises to 44%. An upper limit to the likelihood of 
farmers being highly motivated to join an extension program is predicted to be 25% 
for a scenario in which harvest certainty is raised through an educational program 
and land use is not intense. 

Result of Workshop Participants’ input for Model 2: ‘Effectiveness of Written 
Extension Materials Which are Used as an Extension Aid’ 

The purpose of Model 2 is to supply answers to research question 1 by predicting 
the effectiveness of written extension materials as instructional aids or as ‘stand¬ 
alone’ documents. Using the design of extension material which was developed for 
the program as a guide, the initial model was developed with ‘language’, ‘text 
length’, ‘document length’, ‘graphics’ and ‘type of material’ as parent variables. 
Key measures of effectiveness are farmers’ comprehension of written material and 
the correct construal of meaning. 

Participants accepted the structure of Model 2 and provided data which predicts 
that farmers’ reading ability is affected by both the language in which material is 
presented and the length of text, either phrases or paragraphs (Fig. 3). Participant 
assessments of farmers’ reading ability were that reading skills decline with 


1 The number of trees was nominally set as being at least 50. 
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Fig. 3 BBN model of the effectiveness of written extension materials as stand-alone extension aids 


Table 2 Predicted probability of farmers being able to read either phrases or paragraphs in their local 


dialect, Tagalog or English 

Language (%) 

Text presented 

Text presented 


as phrases 

as paragraphs 

Farmers having ‘high’ reading ability in their local dialect 

97 

62 

Farmers having ‘high’ reading ability in Tagalog 

84 

59 

Farmers having ‘high’ reading ability in English 

58 

38 


increased text length and with departure from the local dialect to the national 
language (Tagalog) or to English. For extension material presented as phrases, the 
predicted probability of farmers having a high level of reading competency—in 
their local dialect (Cebuano or Waray Waray), Tagalog or English—is 97, 84 and 
58%, respectively (Table 2), whereas if material is presented as paragraphs, the 
predicted percentage of farmers having a high level of reading competency declines 
from 62 to 59 and 38%, respectively. These results preclude use of written material 
which is written in other than the local dialect. 

Participants expressed a strong preference for ‘Filipino style’ narrative graphics 
rather than annotated diagrams. Using narrative graphics, and for extension material 
presented as pamphlets, the predicted probability of farmers having a high 
comprehension of extension material declines from 92% if farmers’ reading ability 
is high, to 57% if farmers’ reading ability is low. 

It would have been inappropriate to ask workshop participants about the extent to 
which farmers may misconstrue the meaning of extension information. However, 
previous interviews with farmers in which they had been asked to validate written 
extension materials had indicated that their misconstrual of illustrations concerning 
pruning and thinning was 17 and 33%, respectively (Baynes et al. 2007). For 
scenarios where written extension material is presented as pamphlets, written in 
paragraphs in a local dialect and with narrative graphics, the predicted correct 
construal of extension material when used in the absence of verbal assistance 
declines from 62 (where the misconstrual rate is 17%) to 50% (where the 
misconstrual rate is 33%). 


5) Springer 




















































































Bringing Agroforestry Technology to Farmers in the Philippines 


367 


Result of Workshop Participants’ Input for Model 3: ‘Development of Farmers’ 
Self-Efficacy’ 

Model 3 addresses the need for farmers to develop self-efficacy (research question 
1) and the resource requirements for a scaled-up program (research question 3). 
Workshop participants accepted that farmers’ social confidence during participation 
in extension activities is affected by their social status and their level of education. 
Hence, the principles of adult learning, as described by Knowles (1984), may not be 
applicable to some Filipino farmers because limited education and low social rank 
may inhibit them in extension situations. Consequently, Model 3 was modified so 
that farmers’ social confidence is modelled as a surrogate of farmers’ education 
(Fig. 4). 

In this model, if farmers’ social confidence is high, predicted self-efficacy 
declines with the reduction of inputs from 92% at an optimal state (i.e. individual 
competence, simple theory, extended questions and extended assistance) to 44% 
where inputs are in their most negative state (Table 3). Similarly, if farmers’ 
confidence is low, self-efficacy declines from 79 to only 38% (Table 3). 

A sensitivity analysis was undertaken of the sensitivity of ‘self-efficacy’ to 
variation in ‘prac session effectiveness’, ‘theory session effectiveness’ and 
‘technical reinforcement’. The entropy reduction—the uncertainty of the output 
variable which is expected to be eliminated if the population value of other variables 
are known—was 6.0% for ‘technical reinforcement’, 3.8% for ‘practical session 
effectiveness’ and 2.4% for ‘theory session effectiveness’. These results indicate 



Fig. 4 BBN model of the development of farmers’ self-efficacy through training followed by technical 
reinforcement 
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Table 3 Predicted probability of farmers achieving self-efficacy through the provision of practical and 
theory training and technical reinforcement 


Technical 

reinforcement 

Practical 

competence 

Type of 

theory 

delivery 

Questions 

encouraged 

Farmers with high social 
confidence who achieve 
high self-efficacy 

Farmers with low social 
confidence who achieve 
high self-efficacy 

Yes 

Yes 

Simple 

Yes 

92 

79 

Yes 

Yes 

Simple 

No 

79 

73 

Yes 

Yes 

Complex 

No 

76 

71 

Yes 

No 

Complex 

No 

71 

66 

No 

No 

Complex 

No 

44 

38 



Fig. 5 BBN model of attrition of farmers’ participation in an agroforestry extension program 


that the reinforcement of technical training which is supplied through extended 
on-farm assistance is the most critical success factor for the development of 
self-efficacy. 

Result of Workshop Participants’ Input for Model 4: ‘Attrition of Participating 
Farmers Throughout the Program’ 

Using data which had been collected during the extension program, Model 4 
describes the attrition of participating farmers throughout a program of this nature 
(Fig. 5). The main variable within program managers’ capacity to control is the 
provision of extended assistance. The model provides a response to both research 
question 1 and 2 by predicting the percentage of sites which are planted and for 
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Table 4 Decline in predicted percentage of farmers participating in 

an extension program 

Scenario 

Farmer participation Farmer participation 

with extended assistance (%) with initial assistance (%) 

Encounter severe nursery problems but persist 

68 

7 

Establish trees 

68 

4 

Site survival after 1 year 

61 

3 


which trees survive, depending on the level of extension assistance. The model 
shows a decline in participation because: farmers already know how to establish 
trees; they decide that tree establishment is not feasible; unexpected problems 
occur; they lose interest over a period of time and planted trees are destroyed by 
grazing or fire (Fig. 5). 

Only 77% of farmers who accept an invitation to an introductory field day are 
predicted to initially participate in a program and receive training to begin a home 
nursery. Some farmers are then predicted to lose interest with participation falling to 
68% even if no severe problems occur (Table 4). 

If expected problems are encountered at the nursery stage, and extended 
assistance is not provided, the predicted attrition of participating farmers is very 
severe, only 7% of farmers growing seedlings to the planting-out stage. In addition, 
if encouragement and extended assistance to plant trees are not provided the 
predicted establishment rate of sites is reduced to only 4%. Finally, for this scenario, 
a predicted loss of approximately 10% of sites in the first year reduces the 
percentage of surviving sites to only 3%. Even if farmers do not encounter 
unexpected problems, if encouragement and support are not offered, the predicted 
tree establishment rate is only 36%, a reduction of almost half of participating 
farmers because they gradually lose interest or are distracted by other activities. 


Discussion 

One of the principal advantages of a systems approach to extension evaluation is 
that identification of system variables and assessment of their importance provides a 
rich picture of the socio-economic context in which extension takes place. However, 
in systems approaches involving stakeholder input, there is also the potential for 
participants to accept spurious influences out of a sense of politeness or 
indifference. It is a Filipino characteristic to try to accommodate other viewpoints 
and attitudes (Carson 1978; Goldoftas 2006) and culturally sensitive variables may 
be ignored or glossed over in a workshop situation. 

The ability of BBNs to accommodate imprecise estimates of qualitative variables 
is highly advantageous in extension, which by definition is concerned with 
subjective human values and attitudes. Unfamiliarity with the concept of probability 
can limit the extent to which stakeholders can provide probability estimates and a 
variety of methods have been used to elicit input from them, examples being a 
consensus-based approach followed by expert group review (Henderson and Burn 
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2004) and a sample survey (Cain et al. 2003). Henriksen and Barlebo (2007) used 
stakeholder opinion to populate BBN models of groundwater management, but 
found that explaining BBNs was demanding and time consuming. For this 
workshop, ACIAR staff commented that without the individual assistance supplied 
to every workshop participant, little response may have been elicited from those 
who were less socially-secure. Personal assistance permitted translation and 
repetition of questions to individual participants. ACIAR staff commented that 
this elicited responses which were genuine and considered. 

For program managers, the ability to analyse the sensitivity of program outputs to 
inputs may assist planning and re-direction of resources away from activities which 
are likely to be less successful. In addition, prediction of limits to program outputs 
allows appraisal of the fundamental worth of programs. A difficulty with this type of 
evaluation is that it must be carried out ex poste once the variables have been 
identified and their importance at least tentatively assessed. An ex ante evaluation 
invites model construction based on speculation. For this program, several variables 
(e.g. farmers’ attitudes to fencing or fire control) emerged only late in the program 
and the importance of a key impediment (title security) only emerged at the final 
workshop. 

Usefulness of the Models to Supply Answers to the Three Research Questions 

In providing a response to research question 1, i.e. 4 what extension activities and 
information are appropriate to recruit smallholder farmers in Leyte and to develop 
their self-efficacy so that they are able to continue the establishment of trees without 
further extension assistance’, modelling showed that nursery-based technical 
information was particularly valued by farmers and that personal assistance was 
highly effective in achieving technology transfer. A comparison of the predicted 
levels of contact with farmers for different levels of land-use intensity and political 
favouritism indicated that high levels of these variables are not critical impediments 
to an extension program. Contact with farmers is reduced substantially where 
political cronyism exists. However, participants’ comments—that it is unrealistic to 
expect an extension program to work in a politics-free environment—indicated that 
the influence of this variable may be unavoidable. 

Participants’ insistence that a low level of recruitment will be achieved if title 
security and harvest legislation issues are not resolved, indicated that these issues 
are critical impediments for an extension program of this nature. Strengthening land 
tenure and harvest rights is often described as a prerequisite for increased 
agroforestry adoption, e.g. in Panama (Fischer and Vasseur 2002), in Haiti (Murray 
and Bannister 2004) and in Sumatra (Suyanto et al. 2005). The results of this 
research indicate that in the Philippines, unless land tenure and harvest rights are 
secure, the adoption of agroforestry is unlikely to be adopted beyond its low existing 
level. Whereas farmers’ lack of knowledge of harvesting legislation can be easily 
remedied, title security cannot. Unfortunately the complexities of Filipino law 
preclude a simple remedy for farmers with insecure title. Addressing farmers’ 
awareness of harvesting legislation is therefore a critical success factor for program 
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recruitment and an opportunity which may be exploited in a scaled-up extension 
program. 

As a further response to research question 1, for the predicted 38% of farmers 
who have difficulty reading paragraphs written in their local dialect, the usefulness 
of written extension material is marginal. This is in accord with the findings of 
Kiptot et al. (2006) in Kenya, that technical information must be simplified to help 
farmers’ understanding of complex principles. In this case, even when graphics are 
included to bolster comprehension, approximately one farmer in four is predicted to 
fail to comprehend written materials. The consequences of misconstrual of the 
intended meaning of extension literature are potentially serious. Farmers’ mental 
model of the world may be likened to an invisible force which guides their actions 
(Eckert and Bell 2005). For agroforestry extension in Leyte, this modelling indicates 
that use of written extension materials, if not supported by verbal explanations, is 
ineffective. 

For research question 2, i.e. ‘Will an extension program which offers technical 
advice and assistance be effective in improving the uptake of timber tree 
establishment?’ extension assistance is predicted to be highly effective when 
delivery is optimised to accommodate farmers with limited education and self- 
confidence. Self-efficacy is a precursor to adaption and experimentation with 
technology. Hence, because farmers’ innovations are entry points for technology 
development and adoption (Katanga et al. 2007), variables which affect the 
development of self-efficacy are key drivers of program success. The finding of this 
research—that 35% of farmers may be apprehensive in learning situations—is of 
concern. These farmers are less likely to develop initial self-efficacy and are more 
dependent on reinforcement of skills and knowledge. Even in situations where 
farmers achieve initial competence, only extended assistance may enable them to 
deal with contingencies. Consequently, if unexpected problems occur and extended 
assistance is not provided, the predicted loss of farmers’ participation is predicted to 
be catastrophic. 

It was initially expected that lack of a production to commercialisation (P-C) 
chain for timber trees may constrain farmers’ enthusiasm for extension assistance. 
A well-identified P-C chain provides crops with a distinct adoption advantage 
(Kwesiga et al. 2003; Clement et al. 2004), but in this program farmers had 
previously identified domestic use (to build a house) as a main reason for growing 
trees. For the development of small-scale forestry in Leyte, the development of a 
P-C chain may be an opportunity, but farmers did not identify it as a constraint. 

As a further response to research question 2, the refusal of approximately one 
quarter of farmers who attended an introductory extension activity to then 
participate in an extension program may not be an important indicator of the 
suitability of program offerings. ACIAR staff commented that some farmers may 
enquire about a program to see what material benefits it may bring and a proportion 
of farmers are likely to be unable to come to an arrangement with parents, siblings 
or landlords which will allow them to participate. An early loss of participation of 
farmers from a program may not imply that recruitment methods are at fault. 

Unfortunately, a predicted loss of approximately 10% of sites planted in the first 
year is a serious impediment to the further adoption and diffusion of tree 
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establishment. Negative publicity emanating from the destruction of trees is a 
serious impediment to lifting the level of tree establishment. Fencing is a 
prerequisite if there is a risk of trees being grazed and any form of fire control (e.g. 
liaison with neighbours) is highly advisable. 

A response to research question 3 is collectively provided by responses to 
research questions 1 and 2. For a hypothetically expanded program, a constraint is 
that contact with farmers is only likely to be achieved through elected officials. 
However, an opportunity to maximise recruitment is provided if farmers’ fears of 
being unable to harvest trees in the future are addressed by providing information on 
harvesting legislation. Resource requirements for a hypothetically expanded 
program are predicated by the limited usefulness of written extension material 
and farmer’s preference for extended personal contact with extension staff. Finally, 
the negative impact of farmers’ reaction to the loss of trees through grazing or 
natural disasters represents a system variable over which program managers may 
have little control. 


Conclusion 

For the evaluation of this extension program, systems modelling has provided 
insights into the values and socio-economic conditions of farmers which would have 
been difficult to gain by other means. Use of the BBN modelling approach permitted 
construction of preliminary influence diagrams in which rearrangement of variables 
and causal links could be undertaken until a logical model emerged. Without the 
willing cooperation of workshop participants, validating and populating preliminary 
models would have been very difficult. 

The system variable common to all four models is farmers’ education and 
knowledge, in relation to their understanding of harvest legislation, comprehension 
and construal of written information, development of self-efficacy and their ability 
to cope with problems. Systems modelling highlighted the difficulty of targeting the 
cohort of farmers most in need of extension assistance, namely those with low 
education and social position. If these farmers are not explicitly targeted in program 
design, then the program may mainly benefit richer or better educated farmers. 
Hence, to maximise the effectiveness of a hypothetically expanded program, 
extended assistance and support are necessary. Fortunately, the high predicted 
success rate when extended assistance is provided indicates that programs of this 
nature may present an opportunity to help those farmers most in need. 


Appendix A Definitions of Models and Key Model Variables 

Model la ‘Farmers Contacted by an Agroforestry Extension Program’ 

The model predicts the likelihood of farmers (who are be potentially predisposed to 
growing trees) being contacted and responding to an invitation to join an extension 
program which offers assistance to grow trees. 
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Variable Definition of each state 


LGU contact ‘LGU contact’ is defined as ‘high’ in situations where more than 15 farmers (who may 
potentially be interested in growing trees) per municipality are contacted, invited to be 
included in program activities and respond to the invitation. ‘Low’ contact is defined 
as being contact with less than 15 farmers per municipality. 

Farmers For the definition of this variable, it is assumed that farmers may be interested in 

contacted information about growing and managing trees for a variety of reasons, e.g. planting, 

valuing, application of silviculture, curiosity or the material benefits which the 
program may bring. ‘Farmers contacted’ is defined as ‘high’ in situations where more 
than 15 farmers per municipality are contacted, invited to be included in program 
activities and respond positively to the invitation. ‘Low’ contact is defined as being 
contact with less than 15 farmers and ‘very low’ contact is defined as being contact 
with only one or two farmers per municipality. 


Model lb ‘Farmers Recruited to an Agroforestry Extension Program’ 

The model predicts farmers’ motivation to join an agroforestry extension program. 
It is assumed that for farmers to join the program in a fully participative manner, 
they must have unused land for which there is no alternative use and they must have 
security of harvest. 


Variable Definition 


Potential land 
availability 

Recruitment 


‘Potential land availability’ is defined as being ‘yes’ or ‘no’ depending on whether 
there is sufficient space to grow 50 or more trees, or not, either as a block, rows or 
as inter-plantings. 

‘Recruitment’ is defined as ‘high motivation’ in situations where farmers, after 
being introduced to the program, are motivated to join the program and supply 
necessary inputs to establish trees. ‘Low motivation’ is defined as occurring 
in situations where farmers join the program (often to see what material benefits 
it will bring) but soon lose enthusiasm and abandon their efforts or establish a 
minimal number of trees. ‘No’ recruitment is defined as a decision to not join the 
program. 


Model 2 ‘Effectiveness of Written Extension Materials as an Extension Aid’ 

Model 2 predicts the probability of farmers being able to comprehend (as a personal 
belief) written extension materials. The model also predicts the probability of 
farmers being able of construe the correct interpretation of written material which is 
available in information booths. 


Variable 

Definition 

Graphics 

Language 

‘Graphics’ are defined as being the pictorial accompaniment to text. 

‘Language is defined as being either the local dialect (Cebuano or Waray 
Waray), The Philippines national language (Tagalog) or English. 
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Appendix A continued 


Variable 

Definition 

Reading ability 

‘Reading ability’ is defined as being ‘high’ in situations where farmers 
can read (i.e. understand words and sentence structure) texts of phrases 
or paragraphs. ‘Low’ reading ability is defined as being the ability to 
only read simple well-known instructions and signs only. In this 
situation, farmers are heavily dependent on accompanying graphics. 

Comprehension 

‘Comprehension’ is defined as the meaning which farmers place on text 
or graphics and is defined as being ‘high’ when farmers are able to 
decipher meaning from texts and graphics and ‘low’ when farmers are 
unable or only partly able to decipher a partial meaning from texts and 
graphics. 

Construal 

‘Construal’ is defined as being an interpretation or mental construction 
which is put on text and graphics. Construal is defined as being 
‘correct’ or ‘incorrect’ depending on farmers’ ability to construe the 


correct extension message or not. 

Effectiveness as a stand-alone ‘Effectiveness of a stand-alone extension aid’ is defined as the 
extension aid percentage of farmers who both comprehend written extension 

materials and correctly interpret the extension message 


Model 3 ‘Development of Farmers’ Self-Efficacy’ 

Model 3 predicts the development of farmers’ self-efficacy’—their development of 
self-confidence to the stage where they are confident of growing seedlings and 
establishing trees in the future. 


Variable 

Definition 

General education 

‘General education’ is defined as being either ‘higher than elementary’ or 
‘elementary of lower’. In the first category, farmers can read (i.e. understand 
words and sentence structure) of texts of phrases or paragraphs and can 
undertake simple mathematical computations, (e.g. calculate area from 
measurements of length and breadth). “Elementary or lower” is characterised 
by an inability to perform these functions. 

Individual competence 

‘Individual competence’ is aligned to level 3 of Bloom’s taxonomy for 
psychomotor skills, the ‘yes’ state being achieved when farmers are able to 
repeat (after practice) skills which are shown to them. The ‘no’ state is defined 
as a lack of competence at level 3. The premise underpinning this use of the 
taxonomy is that it was developed by Bloom et al. (1956) and Bloom et al. 
(1971a, b) as a hierarchy of levels of learning behaviour to assist the design and 
assessment of learning activities. For psychomotor skills, level 3 of Bloom’s 
taxonomy is the ability to repeat simple skills with precision, higher levels 
requiring integration of multiple skills. Therefore, we used level 3 as being a 
minimum proficiency level which may lead to the development of self-efficacy. 

Theory-based learning 
activity 

‘Theory-based learning activity is defined as being the supply of underpinning 
theoretical information in an extension program. The ‘simple’ state of this 
variable is achieved at level 2 of Bloom’s taxonomy for cognitive functions in 
which information is understood. The ‘complex’ state of this variable is 
achieved when the delivery of theory exceeds level 2 and requires learners to 
apply, analyse, synthesise or evaluate information. 
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Appendix A continued 

Variable 

Definition 

Self-efficacy 

In a general sense, ‘self-efficacy’ is defined as a personal belief about a 
capability to perform certain actions and exercise influence over life-affecting 
events (Sanna 1992, Bandura 1994). It is an important factor in determining 
subsequent adoption of technology (McGinty et al. 2008). In model 3, self- 
efficacy is defined as being ‘high’ when farmers are sufficiently confident 
about undertaking the activity that lack of knowledge or skills is not an 
impediment. It is defined as being ‘low’ when farmers are not confident about 
undertaking the activity. 

Model 4 ‘Attrition of participating farmers throughout a program’ 

Model 4 predicts the attrition of farmers throughout a program, either because 
establishing trees is not feasible for them, they already have the required knowledge, 
they meet unexpected problems, lose interest or their trees are destroyed by natural 
disasters or grazing. 

Variable 

Definition 

Initial feasibility 

Unexpected 

problem 

‘Initial feasibility’ is defined as being the initial practicality or possibility for farmers 
to establish and grow trees. It is defined as being ‘feasible’ or ‘not feasible 
depending whether farmers have the land, title security, finance and time to 
undertake agroforestry. 

‘Unexpected problem’ is defined as being the occurrence, either ‘yes’ or ‘no’, of an 
event which causes severe unexpected problems which are beyond the capacity of 
farmers to manage. 
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